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SUMMARY 

'T  ; 

This  report  describes  a  free  conversation  opinion 
test,  designed  on  a  statistical  basis,  tn  assess  the  performance 
of  Delta  Sigma  equipments,  connected  both  singly  and  in  tandem. 
Tests  were  made  at  digit  rates  of  19.  2  kb/s  and  12  kb/s.  The 
single  links  were  tested  with  errors  introduced  into  the  digit 
stream,  the  errors  being  either  randomly  distributed  at  a 
constant  density  of  10%  or  in  bursts. 

In  all  cases  the  Delta  Sigma  equipments  were  rated 
higher  on  the  sensitivity/noise  scale  than  the  Army  Standard. 
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1.  INTRODUCTION 


1,  1  The  tests  described  in  this  report  were  carried  out  by  Acoustics 
Group  with  the  object  of  assessing 

(a)  the  performance  of  various  combinations  of  Delta  Sigma 
links  in  tandem 

(b)  the  performance  of  single  Delta  Sigma  links  with  random 
and  burst  errors  introduced  into  the  digit  stream. 

The  circuits  under  test  were  compared  with  the  Army  Standard 
circuit  using  the  Reference  Speech  Link  (RSL)^. 

As  part  of  an  attempt  to  relate  assessments  made  using  the 
free  conversation  test  with  assessments  made  by  other  sub¬ 
jective  methods,  the  test  circuits  were  also  compared  using 
a  modified  rhyme  test^.  Details  of  this  test  are  given  in 
Appendix  A. 

1. 2  The  digital  equipments  were  made  at  S.R.D.  £.  and  are  models 
of  the  GEC  Mk.  3  Gompanded  Delta  Sigma  Modulation  System^. 

1. 3  The  systems  tested  were  as  follows:- 

(a)  Four  wire  speech  circuit  provided  by  the  GEC.  3  Delta 
Sigma  system  working  at  a  digit  rate  of  19.  2  kb/s. 

(b)  Four  wire  speech  circuit  provided  by  the  GEC.  3  Delta 
Sigma  system  working  at  a  digit  rate  of  12  kb/s. 

(c)  Four  wire  speech  circuit  provided  by  the  GEC.  3  Delta 
Sigma  system  working  at  a  digit  rate  of  19.  2  kb/s  with 
bursts  of  50%  errors,  1536  digits  long,  introduced 
randomly  at  an  average  rate  of  one  every  3.  4  seconds 
into  the  digit  stream. 

(d)  Four  wire  speech  circuit  provided  by  the  GEC.  3  Delta 
Sigma  system  working  at  a  digit  rate  of  19.  2  kb/s  with 
bursts  of  50%  errors,  1536  digits  long,  introduced 
randomly  at  an  average  rate  of  one  every  0.  4  seconds 
into  the  digit  stream. 

(e)  Four  wire  speech  circuit  provided  by  three  GEC.  3  Delta 
Sigma  links  connected  in  tandem  through  analogue  inter¬ 
faces,  the  digit  rates  being  19.  2,  19.  2  and  19.  2  kb/s 
respectively. 

(f)  Four  wire  speech  circuit  provided  by  three  GEC.  3  Delta 
Sigmr  links  connected  in  tandem  through  analogue  inter¬ 
faces,  the  digit  rates  being  12,  19.2  and  12  kb/s 
respectively. 
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(g)  Four  wire  speech  circuit  provided  by  two  GEC.  3  Delta  Sigma  • 

links  connected  in  tandem  through  analogue  interfaces,  the 

digit  rates  being  12  and  19.  2  kb/s. 

(h)  Four  wire  speech  circuit  provided  by  12  kb/s  GEC.  3  Delta 
Sigma  system  with  randomly  distributed  digital  errors 
introduced  into  the  digit  stream  at  a  constant  density  of  10%. 

1.4  The  circuits  under  test,  together  with  the  Army  Standard,  were 
rated  with  the  Reference  Speech  Link  in  a  free  conversation  test^. 

2.  EQUIPMENT  SET-UP 

2. 1  All  the  systems  under  test  were  located  in  the  laboratory,  and 
the  handsets  were  extended  into  separate  test  rooms  to  isolate 
them  from  the  equipment  noise. 

2.  2  The  input  level  to  the  digital  system  was  adjusted  just  to  avoid 
peak  clipping  with  a  speech  input  at  mean  normal  talking  level. 

The  optimum  level  was  defined  for  each  digit  rate  as  the  speech 
signal  having  a  peak  amplitude  equal  to  the  peak  amplitude  of  a 
sinusoidal  signal  of  a  1 000  Hz,  at  the  level  at  which  the  input/ 
output  characteristic  just  begins  to  deviate  from  linear  with 
increasing  input  level.  However,  the  optimum  levels  for  the 
two  digit  rates  were  not  greatly  different,  and  the  figure 
appropriate  to  the  12  kb/ sec  digit  rate  was  used  for  all  circuit 
conditions.  Since  the  Delta  Sigma  links  used  were  designed  as 
zero -loss  circuits,  this  avoided  the  need  for  level  matching 
between  links  in  the  tandem  connections.  Input/ output  graphs 
for  a  sinusoidal  signal  of  1000  Hz  are  shown  in  Figure  1. 

2.  3  The  listening  levels  used  for  the  digital  circuits  measured  in 
terms  of  the  signal  voltage  applied  to  the  telephone  receivers, 
were  as  determined  for  a  previous  experiment  by  a  preferred 
listening  level  test^. 

2.4  In  tests  on  tandem  links,  each  individual  duplex  link  was  fed 
from  a  separate  power  supply  and  digit  rate  generator  to  reduce 
the  possibility  of  analogue  speech  breakthrough  or  crosstalk. 

Tests  made  with  optimum  signal  level  applied  to  the  input  of 
one  tandem  channel  only  showed  that  the  signal  level  measured 
at  the  receive  end  of  the  s^me  channel  either  with  any  digital 
signal  path  broken  or  with  the  clock  signal  removed  from  any 
one  demodulator  was  not  larger  than  -50  dB  relative  to  the 
normal  receive  signal.  The  signal  at  the  receive  end  of  the 
other  channel  (near-end  crosstalk)  with  all  connections  intact 
was  -56  dB  relative  to  the  normal  receive  level,  though  any 

crosstalk  here  would  merely  add  to  the  eidetone  signal  during  ' 

actual  tests. 
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2.  5  The  randomly  distributed  digital  errors  were  introduced  into 
the  digit  streams  using  an  S.  R.  D.  E.  designed  generator  of 
random  errors  of  controllable  density^. 

2.  6  The  burst  errors  were  introduced  by  using  gating  circuits  to 
replace  1536-digit  groups  of  traffic  digits  by  random  digits  at 
the  same  bit  rate,  the  interruptions  occurring  randomly  but  at 
controllable  average  rates.  Tests  using  this  form  of  inter¬ 
ruption  were  originally  included  in  connection  with  a  possible 
need  to  introduce  synchronising  patterns  into  the  traffic  stream. 
However,  the  results  may  have  relevance  to  interruptions 
arising  in  other  ways,  e.  g.  intermittent  losses  of  received 
signal.  Siz:=c  any  digit  in  the  random  burst  has  a  50%  prob¬ 
ability  of  having  the  same  state  as  the  digit  in  the  traffic 
stream  which  it  replaces,  the  random  stream  represents  the 
traffic  stream  with  50%  errors.  (100%  errors  would  merely 
invert  the  traffic  stream  and  the  output  waveform). 

2.  7  The  handsets  used  with  all  systems  were  chosen  at  random 

from  a  number  of  instruments  which  had  been  tested  to  specifi¬ 
cation  prior  to  the  experiment.  These  handsets  were  fitted 
with  earphone  element,  magnetic,  Y1/ 5965-99-940-2368.  The 
microphone  for  the  RSL  and  Delta  Sigma  systems  was  the 
microphone  element,  magnetic  Y1/5965-99-901 -1 184  and  for 
the  Army  Standa 'd  system  microphone  capsiile  (carbon)  No.  10, 
YA-7982  was  used. 

2.  8  The  microphone  used  with  the  digital  systems  has  a  rising 
frequency  response  resulting  in  an  output  speech  voltage 
spectrum  which  is  substantially  flat.  This  type  of  input  suits 
the  flat  overload  characteristic  of  the  GEC.  3  Delta  Sigma  system. 

2.  9  The  sensitivity/noise  settings  of  the  RSL^  used  for  the  RSL 

circuit  tests  were  45,  33,  21  and  9  dB.  These  were  thought 
to  have  a  wide  enough  range  to  cover  the  "unknown"  circuits. 

2.10  The  Army  Standard  circuit,  which  consisted  of  two  telephone 
sets  'J*  connected  by  24  miles  of  cable,  electric,  DIO,  had  an 
attenuation  of  36  dB  at  1000  Ha.  An  artificial  cable  designed 
in  sections  of  4,  8  and  12  miles  was  used  in  the  laboratory. 

2.1 1  Details  of  the  test  circuits  are  given  in  Figures  2  and  3. 

3 .  EXPERIMENTAL  PROCEDURE 

3.  1  The  selection  of  13  pairs  of  subjects  for  the  tests  was  facilitated 

by  the  use  of  an  Elliott  920A  computer.  The  computer  was 
programmed  to  select  from  the  S.  R.  D.  E.  civilian  staff  list 
a  group  of  26  members  (22  males  and  4  females)  who  had  not 


3. 


participated  in  a  recent  test  and  whose  age  distribution  was  ' 

similar  to  that  of  the  officer  population  of  the  British  Army 

(Figure  4).  The  final  list  was  arranged  as  11  pairs  of  male 

subjects  and  2  pairs  of  female  subjects.  * 

3.  2  Each  pair  appeared  for  four  separate  sessions  of  approximately 
30  minutes  duration  over  a  period  of  two  weeks,  three  circuit 
conditions  being  presented  at  each  of  three  sessions  and  four 
circuit  conditions  at  the  fourth.  The  subjects  were  given  no 
information  about  the  circuit  conditions  during  the  experiment. 

3.  3  The  final  selection  of  13  circuits  (Figure  5)  consisted  of  four 
settings  of  the  RSL,  the  Army  Standard  and  the  eight  systems 
under  test. 

3.4  The  order  in  which  the  13  circuits  were  presented  to  a  given 
pair  of  subjects  was  determined  from  a  randomised  13x13 
latin  square  (Figure  6),  which  was  obtained  from  statistical 
tables.  This  reduced  systematic  errors  to  a  minimum. 

3.  5  The  subjects  were  housed  in  separate  rooms,  designated  F  and 
G,  having  contact  with  the  operator  via  a  monitoring  system  in 
the  n^ain  laboratory.  Extraneous  noise  was  largely  excluded  p 

from  the  rooms.  Ambient  noise,  typical  of  that  experienced 
in  city  offices'^,  was  artificially  set  up  and  controlled.  The 
two  rooms,  although  acoustically  different  in  normal  use, 
were  modified  with  suitable  materials  to  produce  similar 
ambient  noise  characteristics  at  the  positions  occupied  by 
the  subjects.  The  effects  of  any  remaining  differences  were 
reduced  by  ensuring  that  the  subjects  alternated  between  the 
rooms  during  the  course  of  experiment. 

3.6  The  subjects  were  allowed  to  converse  in  as  free  and  natural 
a  manner  as  the  circuit  condition  permitted.  To  stimulate 
conversation,  the  subjects  were  provided  with  photographs  of 
local  scenes  (Figure  7).  For  each  test,  a  pair  of  photographs 
was  used  showing  the  same  scene  from  the  same  view  point 
but  with  a  short  time  lapse  between  them,  so  that  similarities 
and  differences  could  be  discussed. 

3.7  After  talking  for  about  10  minutes  the  participants  were  asked 
to  express,  separately,  their  opinion  of  the  circuit,  in  terms 
of  the  effort  required  to  conduct  the  conversation.  A  5  point 
scale,  form  8A  (Figure  8)  was  used. 

.'he  opinions  were  subsequently  given  numerical  values  of 
As4,  Bsi,  C=2,  D^l  and  EsO..  an.J  the  results  were  then  passed 
to  Computer  Applications  Group  for  analysis  using  the  Elliott 
920A  Computer. 
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4.  RESULTS  AND  ANALYSIS 


4.  1  The  table  (Figure  9)  shows  the  numerical  scores  obtaineJ  by 
the  subjects  for  each  circuit  condition.  Mean  opinion  scores 
and  statistical  analyses  are  given  in  tables  1  and  2. 

4.  2  The  characteristic  for  the  RSL  (Figure  10)  was  obtained  by 

plotting  the  mean  opinion  scores  against  the  sensitivity/ noise 
^jratios.  The  mean  opinion  scores  for  the  remaining  circuit 
conditions  are  shown  as  intercepts  on  the  RSL  characteristic. 

In  this  fanner  the  rating  for  the  Army  Standard  and  the  'unknown' 
circuits  were  obtained  in  terms  of  sensitivity/noise  settings  on 
the  RSL.  '  These  are  listed  in  Table  1. 
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TABLE  2 
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5.  CONCLUSIONS 


5.  1  In  the  Free  Conversation  Test,  the  digital  systems  were  rated 
in  the  following  order j- 


(1) 

D 

19.  2  kb/s  0  errors 

(2) 

C 

19.  2  kb/s  burst  errors,  1  in  3.4  secs. 

(3) 

B 

19.2  +  19.2  +  19.  2  kb/s  tandem  link. 

(4) 

H 

1 2  kb/s  0  errore 

(5) 

L 

12+  19.2  kb/ s  tandem  link 

(6) 

G 

12  +  19.  2  +  12  kb/s  tandem  link 

(7) 

K 

12  kb/s  10%  constant  errors 

(8) 

I 

1 9.  2  kb/s  burst  errors,  1  in  0.4  secs. 

The  first  six  circuits  were  rated  better  than  "moderate  effort 
required  to  carry  out  a  conversation".  Circuits  K  and  I  came 
out  higher  than  "considerable  effort  required".  All  the 
digital  circuits  were  rated  higher  than  the  Army  Standard. 

Among  the  digital  circuits  tested,  I  and  K  were  the  only  ones 

to  be  given  a  rating  of  "extreme  effort  required,  prolonged  * 

conversation  impossible",  and  in  each  case  this  rating  was 

given  only  once.  Four  "extreme  effort"  ratings  were  recorded 

for  the  Army  Standard.  ' 

5.  2  It  will  be  noted  that  the  ratings  of  circuits  C,  B,  H  and  L  have 
overlapping  confidence  limits,  i.e.  the  order  of  their  ratings 
relative  to  each  other  has  no  statistical  significance.  Similarly 
the  ordering  of  circuits  K,  1  and  the  Army  Standard  is  not 
significant. 

5.  3  The  rating  of  the  Army  Standard  was  markedly  lower  than  in 
the  previous  experiment^. ,  It  has  been  observed  that  the 
absolute  rating  of  a  circuit  repeated  from  experiment  to 
experiment  is  to  some  extent  dependent  on  the  distribution 
relative  to  it  of  the  subjective  standards  of  difficulty  of  the 
remaining  circuits,  i.  e.  the  circuit  is  rated  higher  if  all 
other  circuits  are  considered  more  difficult  and  lower  if  all 
other  circuits  are  considered  easier.  In  the  present  experiment, 
all  the  digital  circuits  were  rated  higher  than  the  Army  Standard, 
whereas  in  the  previous  experiment  more  than  half  the  test 
circuits  were  rated  below  the  Army  Standard.  However,  each 
experiment  is  considered  to  give  a  reliable  ordering  of  the 
circuit  assessments  within  the  limits  of  the  statistical  signifi¬ 
cance.  t 
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APPENDIX  A 


COMPARISON  OF  RHYME  TEST  WITH  FREE  CONVERSATION  TEST 

1.  As  an  attempt  to  relate  assessments  made  using  a  modified  rhyme 
test^  to  those  made  in  the  free  conversation  test,  each  subject  was 
asked,  at  the  conclusion  of  the  conversation  on  each  circuit  condition, 
to  read  out  a  selection  of  the  words  used  in  the  rhyme  test.  These 
words  were  two  and  three  syllable  consonant  and  vowel  combinations 
of  the  form  CV,  VC  and  CVC.  The  words  used  were  arranged  in  104 
groups  of  six,  the  words  in  any  one  group  differing  from  one  another 
by  the  consonant  in  a  constant  end  position  (i.  e.  first  or  last  syllable). 
The  selections  were  chosen  so  that  each  subject  pair  used  the  whole 
sequence  of  groups  during  the  experiment,  allowing  4  groups  to  be 
used  by  each  subject  for  each  circuit  condition.  Similarly  each 
circuit  condition  was  tested  with  every  word  group  during  the  experi¬ 
ment,  so  that  104  decisions  were  made  per  circuit  condition.  In  any 
one  circuit  condition.,  the  speaker  read  out  one  word  from  each  of  the 
four  groups  provided,  the  words  used  having  been  pre -determined  by 
random  selection  (Figure  11).  To  control  variations  in  inflection  to 
some  extent,  each  word  was  embedded  in  the  phrase  "will  you  try.  .  . 
now".  The  words  "try"  and  "now"  were  chosen  for  use  in  the  phrase 
as  having  final  and  initial  syllables  respectively  which  do  not  readily 
form  compounds  with  the  initial  or  final  syllables  of  the  test  word. 

The  listener  selected  and  noted  on  his  copy  of  the  word  list  the  word 
which  he  thought  he  heard. 

2..  The  numbers  of  words  correctly  received  were  logged,  and  Figure  12 
shows  the  scores  obtained  by  each  subject  for  each  circuit  condition. 
These  results  were  passed  to  Computer  Applications  Group  for 
analysis  using  the  Elliot  920A  comput.^r,  and  the  results  are  given 
in  Tables  3  and  4.  The  mean  scores  are  plotted  relative  to  the  RSL 
scores  in  Figure  1  3. 

3.  The  results  of  the  rhyme  test  as  a  means  of  ordering  the  circuit 
conditions  were  disappointing  insomuch  as  the  lowest  mean  subject 
score  was  2.8  words  out  of  4  received  correctly,  whereas  it  had 
been  hoped  that  the  range  of  circuits  would  produce  results  covering 
the  full  range  of  word  scores.  Reference  to  Table  3  shows  that  many 
of  the  differences  between  the  main  scores  for  the  different  circuit 
conditions  are  not  statistically  significant,  so  that  no  reliability  can 
be  attached  to  the  ordering  of  the  circuits  on  the  basis  of  the  scores, 
and  in  fact  this  ordering  is  quite  different  from  that  obtained  from  the 
free  conversation  test.  This  result  suggests  that  insufficient  words 
were  used  in  the  experiment. 

4.  The  high  scores  obtained  in  the  rhyme  test  for  circuit  conditions 
rated  as  difficult  by  the  free  conversation  test  may  arise  because  a 
degree  of  attention  higher  than  that  which  would  be  considered  highly 


10. 


exhausting  for  a  conversation  test  can  be  applied  for  the  short  time 
necessary  to  carry  out  the  word  test.  Analysis  of  the  errors  shows 
that  about  one  third  of  the  speech  sounds  incorrectly  heard  were 
confused  with  sounds  within  the  same  phonetic  groups,  e.  g.  t  with  k, 
m  with  n,  etc.  These  are  confusions  which  can  commonly  occur  in 
the  presence  of  noise.  The  remaining  errors,  which  were  distributed 
over  the  whole  range  of  speech  sounds  used,  appeared  to  be  attributed 
randomly  to  other  speech  sounds  and  probably  contain  a  proportion  of 
guesses.  The  results  suggest  that  the  ratings  of  the  circuits  largely 
reflect  the  varying  degrees  of  quantising  and  error  noise  accompanying 
the  signal,  as  would  be  expected,  rather  than  any  characteristics 
causing  impairments  in  the  received  speech  which  are  selective  in 
their  effect  on  the  various  speech  sounds. 


n 


TABLE  3 


Circuit 

Condition 

Mean 

Correct 
Word 
Score 
out  of  4 

Standard 

Deviation 

Rating 
on  RSL 
Scale 
dB 

95% 

Confidence 

Limits 

+  Correct 
Score 

RSL  45  dB 

3.92 

0.  272 

+  0. 109 

RSL  33  dB 

3.88 

0.  325 

+  0.135 

RSL  21  dB 

3.84 

0.464 

+  0, 187 

RSL  9  dB 

3.  35 

0.628 

+  0. 254 

Army  Standard 

2.96 

1.  14 

5.5  * 

+  0.464 

Delta  Sigma  12  kb/s 

10%  Constant  Errors 

3,04 

0.999 

6.0  * 

+  0.403 

Delta  Sigma  19.  2  kb/s 

C  Errors 

3.46 

0.989 

11.0 

+  0. 399 

Delta  Sigma  12  kb/s 

0  Errors 

3.62 

0.697 

14.0 

+  0. 281 

Delta  Sigma  19.2  kb/  s 
Burst  Errors  1  in 

3.  4  sec. 

3,615 

0.  571 

14.0 

+  0. 231 

Delta  Sigma  19.  2  kb/s 
Burst  Errors  1  in 

0.  4  sec. 

3.  19 

0.633 

7.5  * 

+  0.256 

Delta  Sigma  Tandem  Link 
19.2  +  19.2  +  19.2  kb/s 

3.  12 

0.  746 

6.6  * 

+  0.  349 

Delta  Sigma  Tandem  Link 
12  +  192  +  12  kb/s 

2.81 

0.749 

4.0  * 

+  0. 302 

Delta  Sigma  Tandem  Link 
19.2  +  12  kb/s 

3.  30 

0.  884 

8.9 

+  0. 357 

*  Extrapolated  values 
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Source  of 

Sum  of 

Degrees 

of 

Freedom 

Mean 

Variance 

Significance 
relative  to 

Variance 

Squares 

Square 

Ratio  F 

0.05  significance 
level 

Circuits  ( 1  3) 

41.8 

12 

3.49 

6.  31 

Significant 

Subjects  (26) 

18.85 

25 

.752 

1. 36 

Not  significant 

Residual 

166 

300 

.  553 

Total 


226.65 


337 


.673 
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FIG.  6  LATIN  SQUARE 


I  Opinion  Form  8a 

Opinion  Ratings  based  on  Effort  Required  to 
Conduct  a  Conversation 

Please  express  your  opinion,  when  notified  by  the  operator,  by  ticking  the 
appropriate  letter. 

N,  B,  Do  not  discuss  your  opinion  with  your  companion 

A.  Complete  relaxation  possible;  no  effort  required  . 

B.  Attention  necessary,  no  appreciable  effort  required  . 

C.  Moderate  effort  required  . 

D.  Considerable  effort  required  . 

E.  Extreme  effort  required;  prolonged  conversation  impossible 

Name  Test  No. 

Date 
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Note;  Score  in  top  left  of  each  square  represents  subject  in  room  F 
Score  in  bottom  right  of  each  square  represents  subject  in 

room  G. 


FIG.  9  TABLE  OF  RESULTS  OF  FREE 
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FIG.  1 1  EXAMPLE  OF  RHYME  TEST  WORD  CARDS 


Note;  Score  in  top  left  of  each  aquare  >reaenta  number  of 
correct  worde  scored  out  of  four  in  room  F. 


Score  in  bottom  right  of  each  square  represents  number  of 
correct  words  scored  out  of  four  in  room  G. 


FIG.  12  TABLE  OF  WORX)  RESULTS 
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